/*
The analyses are organized for the specific results discussed
in the section titled: 

Do pilot studies and attention or manipulation checks improve significance rates?
*/

use "../Metadata/tess_analysisdata.dta", clear
 
** Pilot data

	* most studies include pilot data - 76%
	ta  pilot if hyp_num == 1

	* rate of supported hypotheses lower for studies with pilot data - 20.9 vs. 30.1%
	prtest hyp_true if insample == 1, by(pilot)
	
	* studies with pilot data less likely to have positive results -- 52.6 vs. 58.3%
	prtest successfulexp_insample if hyp_num == 1, by(pilot)



** Manipulation/Attention/Comprehension checks

	* how many studies used checks -- 30%
	tab any_checks if hyp_num == 1

	* rates of positive findings at study-level -- 50 vs. 55.7%
	prtest successfulexp_insample if hyp_num == 1, by(any_checks)

	* rates of supported hypotheses -- 20.5 vs. 25.5
	prtest hyp_true if insample == 1, by(any_checks)

	* sample size differences in whether experiments used checks -- p=.07
	ttest samplesize if hyp_num==1, by(any_checks)

** Completion rates

	destring completionrate, replace

	* study-level beta = -.012
	reg successfulexp_insample completionrate i.vendor if hyp_num==1

	* hypothesis-level analysis beta = -.003
	reg hyp_true completionrate i.vendor if insample==1, vce(cluster vendor_id)

